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ABSTRACT 

An evaluation was conducted of th** first year of a 
3~year program designed in New York City to serve the needs of 
Chinese-speaking intermediate grade students ot limited English 
proficiency. The focus of the program was on improving their 
achievement in mathematics and science while also improving students* 
English language proficiency. The project proposed to provide 
enriched mathematics and science instruction in Chinese during the 
school year and in a 5-week summer institute, with workshops at the 
school district's Mathematics and Science Resource Center, The 
program admitted students who had scored at or below the 40th 
percentile on the Language Assessment Battery, The project enrolled 
230 male and 185 female students in grades 4 through 6, of whom 97,8 
percent were eligible for free lunches. For the most part, the 
program carried out the activities proposed in its design. The 
children were observed to enjoy the resource center activities in 
particular. The project met its objectives in English language 
proficiency and improvement of mathematics and l.igher order thinking 
skills. Objectives in Engl ish-as-a-Second-Language reading were not 
met. Objectives were met with regard to parent understanding of the 
children's mathematics and science education, but objectives for 
parent participation at activities scheduled for them were not met. 
Recommendations for program improvement in the remaining 2 years of 
federal funding are suggested. Appendixes describe data collection 
and analysis and instructional materials, (SLD) 



CD 
ITS 

col 



SCOPE OF INTEREST NOTfCE 1 

Tho ERIC Facility hat assigned 
this documvnt for processing 

to: 



In our judgment, this document 
is also of interest to the Clear- 
inghouses noted to the right. 
Indexing should reflect their 
special poir^ts of view. 



3/ 




M OREA Report 



Chinese Bilingual Math and Science 
Community School District. 2, Manhattan 
iTransitional Bilingual Education Grant Number T003A10191 

1991-92 

FINAL E\'ALUATION PROFILE 



nn " ?,''^''*"MENT OF EDUCATION 

^Jil^rfT''"' '^^ reproduced as 

S:t"rodg_ora?af;; '° ~ 

' mPn/^°' °' stated .n th.s docu. 

SIS on<?n "««=^^-'v represent oM^.ai 

UtHI Position or policy 



I "PERMISSION TO REPRODUCE THIS 
MATERIAL BEEN GRANTED BY 



I TO THE EDUCATIONAL RESOURCES 
I INFORMATION CENTER (ERIC) ' 



Chinese Bilingual Math and Science 
Community School District 2, Manhattan 
Transitional Bilingual Education Grant Number T003A10197 

1991-92 

FINAL EVALUATION PROFILE 



NEW YORK CITY BOARD OF EDUCATION 



H. Carl McCall 
President 

Irene H. Impellizzeri 
Vice President 

Carol A. Gresser 
Wesiina Matthews 
Michael J. Petrides 
Luis O. Reyes 
Ninfa Segarra 
Members 

Keysha Z. McNeil 
Student Advisory Member 



Joseph A. Fernandez 
Chancellor 



DIVISION OF STRATEGIC PLANNING/RESEARCH & DEVELOPMENT 

Robin Wiilner 
Executive Director 



II is the policy of the New Yock Cily Botrd of Educjiion noi lo disciiminalc on the buis of nee, color, creed, religion, 
naiional oii(pn, age, handicapping condition, marital sutut. aexual orienution. or tex in iu educational programs, 
activities, and employment policies, and to maintain an environment free of aexual haiaiimenl, u xequijed by law. 
Inquiria regarding compliance with appropriate lawa may be directed to Mercedet A. Neaficld, Director, Office of 
Equal Opportunity, 110 Uvingitai Stitjct, Room 601. Brodayn, New Yoik 1 1201, Telephone: (718) 935*3320. 



2/93 



ACKNOWLEDGMENTS 



Tliis report has been prepared by the Bilingual, Multicultural, and Early Childhood Evaluation Unit of 
the Office of Research, Evaluation, and Assessment. Thanks are due to Joanne Lynch for collecting the data and 
writing the report. 

Additional copies of this report are available from: 

Dr. Tomi Deutsch Barney 

Office of Research, Evaluation, and Assessment 

New York City Public Schools 

110 Livingston Street, Room 732 

Brooklyn, NY 11201 

(718) 935-3790 FAX (718) 935-5490 



r 

kJ 



FOREWORD 



The body of this report is preceded by an Extract which presents an overview of sah'ent points of 
the project: funding cycle; enrollment figures; background of students served; admission criteria; and 
programming features, strengths, and limitations, inclur!ing the outcome of all objectives. The extract also 
presents the conclusions drawn by the Office of ReseaK , , Evaluation, and Assessment (OREA) about the 
program and its recommendations for program improvement. 

The extract is followed by the body of the report, titled Program Assessment . This includes such 
information as staffing, program implementation, and outcome and implementation objectives, 
instructional objectives are presented first, followed by noninstructional objectives. The report then 
addres:;es those aspects of programming mandated by Title VII regulations that do not have specifically 
stated objectives. This may be information on attendance and dropout rate, grade retention, 
mainstreaming, referrals out of the program to meet special needs of the students, and withdrawals. A 
case history concludes the report. 

Data for this profile were collected and analyzed using a variety of procedures, which are 
described in A ppendix A following the text. 
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Office of Research, Evaluation, and Assessment 
New York City Public Schools 
E.S.E.A. Title VII Evaluation Profile 

Chinese Bilingual Math and Science 

Community School District 2, Manhattan 
Transitional Bilingual Education Grant T003A10197 

1991-92 

EXTRACT 

PROJECT COORDINATOR: Ms. Kit Fung FUNDING CYCLE: Year 1 of 3 

SITES 



Schcx)l 




Grade Levels 


Enrollment* 


P.S. 


1 


4-6 


84 


P.S. 


2 


4-6 


114 


P.S. 


42 


4-6 


77 


P.S. 


124 


4-6 


56 


P.S. 


130 


4-6 


84 



*The project enrolled 415 students. Male students numbered 230» female 185. 



STUDENT BACKGROUND 



Native Language Number of Students Countries of Origin Number of Students 

Chinese 399 China 341 

Mandarin 87 Hong Kong 48 

Cantonese 192 Ivlalaya 6 

Other 120 United States 5 

Unreported 12 Taiwan 2 

Other 7 

Unreported 6 



Median Years of Education in Native Country: 3»0; in the United States: 2.0 
Percentage of Students Eligible for Free Lunch Program: 97.8 



ADMISSION CRITERL\ 

The Chinese Bilingual Math and Science program admitted students who had scored at or below the 40th 
percentile on the Language Assessment Battery (LAB) and were thus classified as being of limited English 
proficiency (lEP). 



PROGRAMMING 
Design Features 

The Chinese Bilingual Math and Science project was designed to serve the needs of Chinese-speaking 
LEP students. The focus of programming was on improving students' achievement in mathematics and science 
while also improving their EngHsh language proficiency. The project proposed to provide enriched mathematics 
and science instruction in Chinese during the school year and at a five week summer institute. Staff attended 
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workshops every month at the Community School Dislrict*s (C.S.D/s) Mathematics and Science Resource Center. 
After each workshop, project students returned with the teachers to the resource center and participated in 
hands-on and experiential activities that augmented classroom instruction. 

Capacity building . After three years of federal funding, C.S.D. 2 will support the project with tax levy 
and other funds. 



Strengths and Limitations 

For the most part, the Chinese Bilingual Malh and Science program carried out the activities proposed in 
its design. Project staff noted that the children particularly enjoyed the resource center. The materials provided 
through the project were exemplary. 



CONCLUSIONS AND RECOMMENDATIONS 

In its first year of programming, the project met its E.S.L. objectives for English language proficiency, 
improvement in mathematics skills, mastery of mathematics, high order thinking skills related to mathematics, 
science, student career awareness, staff development, and parental understanding of and participation in their 
child's mathematics and science education. The project did not meet its objectives for E.S.L. readmg and 
attendance at parental activities. The Office of Research, Evaluation, and Assessment (OREA) could not 
evaluate the objective for the Citywide Science exam as currently stated. 

Ttie conclusions, based on the findings of this evaluation, lead to the following recommendations: 

• Explore reasons why students* achievement of English language skills as measured by the Degrees of 
Reading Power (D.R.P.) test was less than anticipated and take steps to rectify the situation in the 
future. 

• Attempt to increase attendance at parent activities. 

• Seek permission to replace the objective for the Citywide Science exam. 
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PROGRAM ASSESSMENT 



STAFFING 

Title VII Staff (Total 2) 
Title 

Resource Specialist 
Resource Specialist 



D egree 

MA. 
MA. 



Language Competencies Title VII Funding 



Cantonese 
Cantonese 



Full time 
Full time 



Other Staff WorKing With Project Students (Total 14) 
Title Degrees Certification 

Teacher 14 M.A. 13 Bil. Chinese Common Branches 6 
BA. 1 Bil. Chinese Com. Branches PPT 1 

Bil. Common Branches TPD 2 

Bilingual PPT 1 

E.S.L. Common Branches 1 

Bilingual Chinese 1 

Common Branches DES 2 

Chinese Bilingual Ancillary 1 

E.S.L. Ancillary 2 

Mandarin 2 

Cantonese 1 

All staff had certification in areas they served. 



Language Competencies and Teaching/ 
Communicative Proficiencies (TP/CPV 

Chinese TP 13, CP 1 



IMPLEMENTATION AND OUTCOMES (Objectives prefaced by •) 

English as a Second Language (E.S.L.) 

Students received five periods of E.S.L. per week. Techniques of instruction included such activities as 
creating books, sharing stories, and using hands-on manipulatives in a whole-language approach. 

See Appendix B for a list of the instructional materials used. 

• Participating LEP students will have shown improvement on the English language version of the l AB 
greater than that of a non-project comparison group. 

• Participating students will improve their English language skills as demonstrated by a significant 
increase in scores on the Language Assessment Battery (LAB), as measured by a correlated r-test for 
significance of difference between pre- and posttest scores converted into Normal Curve Equivalent 
(N.C.E.) units. 



^Teaching Proficiency (TP): Competent to teach in this language. 
Communicative Proficiency (CP): Conversational capability only. 
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Evaluation Inslrumenl: Language Assessment Battery (LAB)* 
Pretest: May 1991; postlcsl: May 1992. 

Number of students for whom pre- and postlesl data were reported: 289 

Percent of students with pretest/posllesl gains: 64.4 

Mean gain: 7.5 N.C.E.s (s.d.=9.2) 

Mean gain is statistically significant (/ = 13.85, /?<. 05). 

Project met both E.S.L. objectives for English lanj^uuge pronciency. 

• Participating students will improve their English language skills as demonstrated by a significant 

increase in scores on the Degrees of Reading Power (D.R.P.) test, as measured by a correlated /-lest 
foi significance of difference between pre- and posltest scores converted into Normal Curve 
Equivalent (N.C.E.) units. 

Evaluaiion Instrument: Degrees of Reading Power (D.R.P.) 
Pretest: May 1991; posttesl: May 1992. 

Number of students for whom pre- and posttest data were reported: 44 

Mean gain: 1.14 N.CE.s (s.d.=9.4) 

Mean gain is not statistically significant (/ = .80, /)>.05). 

Project did not meet E.S.L. reading objective. Tlie project director indicated that additional staff 
development regarding effective ways of teaching E.S.L. will be given in the 1992-93 school year. 

Native Language Arts TN.LA.) 

Students lacking native language literacy skills (estimated): 15 percent. 

Students received five periods of N.L.A. instruction per week. The teaching method used was total 
physical response, which incorporated the translation of key vocabulary, concepts, and lessons in Chinese and in 
EngUsh. 

See Appendbc B for a list of the instructional materials used. 



*OREA used a gap reduction design to evaluate the effect of supplementary instruction on project students* 
performance on the LAB. Since all LEP students in New York City are entitled to such instruction, no valid 
comparison group exists among these students, and OREA used instead the group on which the LAB was 
normed. Test scores are reported in Normal Curve Equivalents (N.C.E.s), which are normalized standard scores 
with a mean of 50 and a standard deviation of 21.06. It is assumed that the norm group has a zero gain in 
N.C.E.S in the absence of supplementary instruction and that participating students' gains are attributable to 
project services. 
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Content Area Subjects 



Mathematic s, Students received five periods of mathematics instruction each week. Teachers emphasized 
hands-on and manipulative techniques and games. For those project students on the lowest level of English 
proficiency, instruction was in Chinese. As they became more proficient, instruction progressed to EnglisL 
supplemented by Chinese, and finally to English with an E.S.L. approach. 

• Participating students will improve their mathematics skills as demonstrated by a significant increase 
in scores on the Metropolitan Achievement ^ est (MAT) in mathematics, as measured by a correlated 
/-test for significance of difference between pre- and posttest scores converted into Normal Curve 
Equivalent (N.CE.) units. 

Evaluation Instrument: Metropolitan Achievement Test in Mathematics (MAT-Math) 

Pretest: April 1991; posttest: April 1992. 

Number of students for whom pre- and posttest data were reported: 125 

Mean gain: 9.46 N.CE.s (s.d. = 13.5) 

Mean gain is statistically significant (r=7.84, p<.QS). 

Project met objective for improvement in mathematics skills. 

• At least 75 percent of participating students will demonstrate a mastery of mathematics by scoring 80 
percent or higher on a criterion-referenced district or teacher-developed Chinese language 
mathematics test. 

Evaluation Indicator: final course grades. (Final course grades had to be used since Chinese language 
mathematics test was not available.) 

Percent of students receiving final grades of at least 80: 88 
Project met objective for mastery of mathematics. 



• At least 75 percent of students will demonstrate acquisition of high-order thinking skills related to 
mathematics. 

Evaluation Instrument: Problem Solving subtest of the MAT-Math. 

Number of students for whom pre- and posttest data were reported: 125 

Mean gain: 8.66 N.CE.s (s.d.=16.46) 

Mean gain is statistically significant (/=5.88, /?<.05). 

Project met objective for high order thinking skills related to mathematics. 

Science . Students received three periods of science instruction weekly. As English proficiency increased, 
project students received less instruction in Chinese and more in English until classes were entirely in English 
using an E.S.L. methodology. Instruction included the use of hands-on activities. Students created natural 
habitats such as a butterfly nest and an ant farm, and used prisms, magnets, batteries, etcetera. 

See Appendix B for a list of the instructional materials used. 
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• Participating students will improve their science skills, as demonstrated by a significant increase in 
scores on the Citywide Standardized Science Achievement Test. 

Tliis objective could not be measured as stated since only fourth graders take the ex^m» and it is only 
given once, precluding any measurement of improvement. 

Objective for Citywide Science Achievement Test couid not be measured. ORE A recommends that this 
objective be revised for the 1992-93 school year. 

• Seventy-five percent of students will score 80 percent or better on a criterion-referenced district or 
teacher-developed Chinese language science test. 

Evaluation Indicator: final course grades. (Final course grades were used since a Chinese language science test 
was not available). 

Percent of students receiving a grade of at least 80: 89 
Project met objective for science. 

Career Awareness 

Students involved in the project received information designed to make them more aware of career 
opportunities in the fields of mathematics and science. A Career Awareness Day was held at P.S. 1 and students 
from all schools were invited. 

• Seventy-five percent of students will demonstrate acquisition of information about mathematics and 
science-related education, training, and employment options. 

Evaluation Indicator: OREA-devcloped questionnaire to measure knowledge of educational requirements and 
employment opportunhies in mathematics- and science-related fields. 

OREA developed a questionnaire which required students to list jobs related to mathematics and science 
and what type of education was needed for these jobs. Ninety-five percent of students demonstrated knowledge 
of educational requirements and career opportunities in mathematics and science. 

Project met objective for career awareness. 



Attendance 

No attendance objective was proposed. However, attendance rates are listed below. 



School Project Students Attendance Rate Mainstream Students 

P,S. 1 96.7 92.6 

P.S. 2 99.0 94.4 

P.S. 42 98.1 95.7 

P.S. 124 98.0 96.2 

P.S. 130 98.8 94.9 



At each school, project students* attendance was higher than that of the mainstream students. 



Other Activities 



Project students participated in various trips over the year, including a camping .rip to Gateway National 
Park, and visits to the New York Hall of Science and the Brooklyn Academy of Music. 

Students With Special Academic Needs 

Referral to special education. Teachers referred students whose ability to function in the program was in 
A doubt to a School-based Support Team (S.B.S.T.). The educational evaluator and the counselor were bilingual. 
No students were referred to special education. 

Remedial programs. No students were referred to remedial programs. 

Gifted and talented programs. The project offered special computer programs to children who were 
gifted or talented. No students were referred to prograrns for the gifteH and talented. 

Staff Development 

The project held staff development workshops for project teachers every month at the district Science 
Resource Center. Topics included: "Science in the Elementary School," "Water in Our Lives," "Llectricity - 
Methods and Motivation," "Insect-Life Cycle," "Go Take a Hike," and "Your Place in Nature." The resource 
specialists supervised the teachers when they later presented this material to their classes. 

An OREA-developed questionnaire examined whether participating teachers increased their knowledge of 
and their ability to teach bilingual mathematics and science. The questionnaire also asked teachers to rate their 
satisfaction with the project curriculum. 

• At least 80 percent of participating teachers and paraprofessionals will have demonstrated an 
increased knowledge of bilingual mathematics and science content and methodology, as measured by 
pre- and post-training questionnaires. 

All the participating teachers indicated that their knowledge of bilingual mathematics and science content 
and methodology increased as a result of the workshops. 

Project met objective for increasing staff knowledge. 

• At least 80 percent of participating teachers and paraprofessionals will have improved their ability to 
use project content and methodology, as measured by pre- and post questionnaires and by structured 
classroom observations by the program director, staff developers, and an independent evaluator. 

All the participating teachers indicated that their ability to use project content and methodology improved 
as a result of the workshops. 

Project met objective for improvement in staff ability. 



• At least 80 percent of the participating teachers will find satisfactory for use with their students 
project adapted curriculum, as measured by a questionnaire. 

All the participating teachers indicated that they found the project curriculum satisfactory. 

Project met objective for staff satisfaction with curriculum. 
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Summer Institute 



A summer institute was held for staff for one week in August. Staff were given training in hands-on 
activities to use in teaching mathematics and science. 

• By August 31, 1992, at least 80 percent of teachers and paraprofessionals participating in the summer 
training institute will have demonstrated an increased knowledge of bilingual mathematics and science 
content and methodology, as measured by pre- and post-training questionnaires. 

An OREA-developed questionnaire examined whether staff participating in the summer training institute 
increased their knowledge of bilingual mathematics and science content as well as methodology. All the 
participating staff indicated that their knowledge of bilingual mathematics and science content and methodology 
increased as a result of the training they received at the summer institute. 

Project met objective for summer institute. 
Parental Involvement 

• At least 60 percent of parents of participating students will have taken part in three or more activities 
for parents, as measured by attendance records for parental involvement activities. 

Parents of project students attended various activities throughout the school year. The project director 
provided the following attendance rates for activities: Advisory council meetings 30 percent, open-school 
day/evening 40 percent, parent-teacher conferences 90 percent, and Citywide Chinese Parents Conference 80 
percent. 

Project did not meet objective for parental involvement. The project director indicated that for the 
1992-93 school year, advisory council meetings will be held in the schools involved in the project instead of the 
district office so that more parents will be able to attend. In addition, staff developers will accompany parents if 
they have to travel to another site to attend activities. 

• At least 60 percent of the parents of participating students will have demonstrated a satisfactory level 
of understanding of and participation in their children's mathematics and science education, as 
mcc^oured by a parent checklist in English and Chinese and by teacher reports. 

Parents received an OREA-developed English and Chinese language questionnaire to assess their 
understanding of and participation in their children's mathematics and science education. All the respondents 
indicated that participating in the program helped them to understand their child's mathematics education. 
Ninety-nine percent indicated that participating in the program helped them to understand their child's science 
education. Ninety-nine percent indicated that, as a result of the program, they have participated more in their 
child's mathematics education. Ninety-six percent indicated that as a result of participating in the program they 
have participated more in their child's science education. 

Project met objective for parental understanding of and participation in their child's mathematics and 
science education* 



CASE HISTORY 

The project did not provide a case history. 
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APPENDIX A 



DATA COLLECTION AND ANALYSIS 



COLLECTION 

OREA evaluation consultants visit sites and interview key personnel. The project director gathers data 
and, with the consultant, completes forms (as shown below) as necessary. 

Student Data Form 

This one-page form is filled out by staff for each participating and mainstreamed student. OREA gathers 
data from this form on backgrounds, demographics, academic outcomes, attendance, referrals, and exit from the 
program. 

Project Director's Questionnaire 

The Project Director's Questionnaire includes questions on staff qualifications, program implementation, 
periods of instruction, and instructional materials and techniques. 

Project Director's Interview 

The interview gathers information on program and student or staff characteristics not supplied by the 
Project Director's Questionnaire. The interview also allows project staff to offer qualitative data or amplify 
responses to the questionnaire. 

Citywide Test Scores 

OREA retrieves scores centrally from the Language Assessment Battery (LAB) and other citywide tests. 
For evaluation purposes, these test scores are reported in Normal Curve Equivalents (N.C.E.s). N.C.E.s are 
normalized standard scores with a mean of 50 and a standard deviation (s.d.) of 21.06. They constitute an equal- 
interval scale in which the distance is the same between any two adjacent scores. A gain of 5 N.CE.S is the same 
whether it is at the lower or the higher end of the scale. N.C.E.s can be used in arithmetic computations to 
indicate group progress. (Percentile scales, although more familiar to many, are unsuitable for such computations 
since they are not equal-interval.) 

Likert-Tvpe Surveys 

Likert-type surveys, in which respondents mark their opinions on a scale from one to five, are used in a 
variety of ways. They examine student attitudes (i.e., toward school and career, native language use, and native 
and mainstream cultures). They also assess staff and parent attitude and reactions to workshops and other 
activities. 



ANALYSIS 

Gap Reduction Evaluation Design 

OREA uses a gap reduction design for measuring changes in standardized tests. Since no appropriate 
non-project comparison group is available in New York City, where all students of Umited EngUsh proficiency 
(LEP) are entitled to receive supplementary services, OREA compares the progress of participating students with 
that of the group oc which the test was normed. It is assumed that the norm group would show a zero gain in 
the absence of instruction, and gains made by project students could be attributed to project services. Scores are 
reported in Normal Curve Equivalents (N.CE.s), which are normalized standard scores with a mean of 50 and a 
standard deviation of 2L06. (See "Citywide Test Scores" above.) 

To test whether pre/posttest gains are greater than could be expected by chance alone, OREA uses a /- 
tesv. To test whether a difference between two proportions (e.g., program and mainstream attendance rates) is 
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greater than could be expected by chance, OREA uses a z-test and reports the differences between the two 
proportions. The level of significance is set at .05 for all tests. 

Techniques For Minimizing Error 

The evaluation procedures minimize error by providing for proper administration of evaluation 
instruments through a combination of testing at 12-month intervals, appropriate analysis procedures 
and reporting. 

Instruments of measurement include the LAB (see above), the Degrees of Reading Power (D.R.P.) test, 
the Metropolitan Achievement Test-Mathematics (MAT-Malh), El Examen de Lectura en Espanol (ELE), 
Likert-type scales (see above), and project-developed tests. Except for Likert scales and project-developed tests, 
these instruments are scored on a cirywide basis at the Scan Center of the New York City Public Schools. 
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APPENDIX B 
INSTRUCTIONAL MATERIALS 



E.S.L. 



Tile Four Elements 

The Four Senses 

The Four Seasons 

Life on Earth 

Aims Publications 

Science in the Content Area 

Sunshine Science Series 



Greenshower^ 1990 

Greenshower, 1990 

Greenshower, 1990 

Greenshower, 1990 

Aims Education Foundation, 1987 

Rigby, 1990 

Wright Group, 1990 



The Four Elements 
The Four Senses 
The Four Seasons 
Life on Earth 



Greenshower, 1990 

Greenshower, 1990 

Greenshower, 1990 

Greenshower, 1990 



Mathematics 

AIMS'Adventures in Math and Science 
Game 24 

Chinese Mathematics 

Progress in Math 
Mathematics 



R. Sun 

D. Gundlach, K. Buffie 
Duncan, Quast, Capps, Ebos 



Aims Education Foundation, 1987 
Suntex International, 1989 
Evaluation Dissemination and 

Assessment Center 
Laidlaw Brothers 
Houghton Mifflin Company 
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